Script-based Multimodal Output Generation for a Conversational Character

This paper reports on the generation of coherent multimodal output of an embodied conversational character in a 3D game setting. In our implementation, we employ a hierarchical two-tiered approach that supports the designer in creating agent behaviors through scripts. Lower-level scripts define sets of both elementary animations and custom animations, along with their temporal duration. At a higher level, sequences of elementary or custom animations are used to define more complex behaviors of variable time duration. At each hierarchical level, additional scripts of syntactic rules take care of specifying smooth transitions from one animation and to the next one. The use of placeholders within scripts allows for parts of the non-verbal output to be selected at run-time thus increasing variety in behavior display. The English-like language chosen for the scripts makes the system very versatile and easy to use. The generation system works in real-time and any user can easily add behaviors at will.